91 research outputs found
Learning stable and predictive structures in kinetic systems: Benefits of a causal approach
Learning kinetic systems from data is one of the core challenges in many
fields. Identifying stable models is essential for the generalization
capabilities of data-driven inference. We introduce a computationally efficient
framework, called CausalKinetiX, that identifies structure from discrete time,
noisy observations, generated from heterogeneous experiments. The algorithm
assumes the existence of an underlying, invariant kinetic model, a key
criterion for reproducible research. Results on both simulated and real-world
examples suggest that learning the structure of kinetic systems benefits from a
causal perspective. The identified variables and models allow for a concise
description of the dynamics across multiple experimental settings and can be
used for prediction in unseen experiments. We observe significant improvements
compared to well established approaches focusing solely on predictive
performance, especially for out-of-sample generalization
Invariant Causal Prediction for Sequential Data
We investigate the problem of inferring the causal predictors of a response
from a set of explanatory variables . Classical
ordinary least squares regression includes all predictors that reduce the
variance of . Using only the causal predictors instead leads to models that
have the advantage of remaining invariant under interventions, loosely speaking
they lead to invariance across different "environments" or "heterogeneity
patterns". More precisely, the conditional distribution of given its causal
predictors remains invariant for all observations. Recent work exploits such a
stability to infer causal relations from data with different but known
environments. We show that even without having knowledge of the environments or
heterogeneity pattern, inferring causal relations is possible for time-ordered
(or any other type of sequentially ordered) data. In particular, this allows
detecting instantaneous causal relations in multivariate linear time series
which is usually not the case for Granger causality. Besides novel methodology,
we provide statistical confidence bounds and asymptotic detection results for
inferring causal predictors, and present an application to monetary policy in
macroeconomics.Comment: 55 page
Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise
We introduce coroICA, confounding-robust independent component analysis, a
novel ICA algorithm which decomposes linearly mixed multivariate observations
into independent components that are corrupted (and rendered dependent) by
hidden group-wise stationary confounding. It extends the ordinary ICA model in
a theoretically sound and explicit way to incorporate group-wise (or
environment-wise) confounding. We show that our proposed general noise model
allows to perform ICA in settings where other noisy ICA procedures fail.
Additionally, it can be used for applications with grouped data by adjusting
for different stationary noise within each group. Our proposed noise model has
a natural relation to causality and we explain how it can be applied in the
context of causal inference. In addition to our theoretical framework, we
provide an efficient estimation procedure and prove identifiability of the
unmixing matrix under mild assumptions. Finally, we illustrate the performance
and robustness of our method on simulated data, provide audible and visual
examples, and demonstrate the applicability to real-world scenarios by
experiments on publicly available Antarctic ice core data as well as two EEG
data sets. We provide a scikit-learn compatible pip-installable Python package
coroICA as well as R and Matlab implementations accompanied by a documentation
at https://sweichwald.de/coroICA/Comment: equal contribution between Pfister and Weichwal
Supervised Learning and Model Analysis with Compositional Data
The compositionality and sparsity of high-throughput sequencing data poses a
challenge for regression and classification. However, in microbiome research in
particular, conditional modeling is an essential tool to investigate
relationships between phenotypes and the microbiome. Existing techniques are
often inadequate: they either rely on extensions of the linear log-contrast
model (which adjusts for compositionality, but is often unable to capture
useful signals), or they are based on black-box machine learning methods (which
may capture useful signals, but ignore compositionality in downstream
analyses).
We propose KernelBiome, a kernel-based nonparametric regression and
classification framework for compositional data. It is tailored to sparse
compositional data and is able to incorporate prior knowledge, such as
phylogenetic structure. KernelBiome captures complex signals, including in the
zero-structure, while automatically adapting model complexity. We demonstrate
on par or improved predictive performance compared with state-of-the-art
machine learning methods. Additionally, our framework provides two key
advantages: (i) We propose two novel quantities to interpret contributions of
individual components and prove that they consistently estimate average
perturbation effects of the conditional mean, extending the interpretability of
linear log-contrast models to nonparametric models. (ii) We show that the
connection between kernels and distances aids interpretability and provides a
data-driven embedding that can augment further analysis. Finally, we apply the
KernelBiome framework to two public microbiome studies and illustrate the
proposed model analysis. KernelBiome is available as an open-source Python
package at https://github.com/shimenghuang/KernelBiome
Identifying Representations for Intervention Extrapolation
The premise of identifiable and causal representation learning is to improve
the current representation learning paradigm in terms of generalizability or
robustness. Despite recent progress in questions of identifiability, more
theoretical results demonstrating concrete advantages of these methods for
downstream tasks are needed. In this paper, we consider the task of
intervention extrapolation: predicting how interventions affect an outcome,
even when those interventions are not observed at training time, and show that
identifiable representations can provide an effective solution to this task
even if the interventions affect the outcome non-linearly. Our setup includes
an outcome Y, observed features X, which are generated as a non-linear
transformation of latent features Z, and exogenous action variables A, which
influence Z. The objective of intervention extrapolation is to predict how
interventions on A that lie outside the training support of A affect Y. Here,
extrapolation becomes possible if the effect of A on Z is linear and the
residual when regressing Z on A has full support. As Z is latent, we combine
the task of intervention extrapolation with identifiable representation
learning, which we call Rep4Ex: we aim to map the observed features X into a
subspace that allows for non-linear extrapolation in A. We show using Wiener's
Tauberian theorem that the hidden representation is identifiable up to an
affine transformation in Z-space, which is sufficient for intervention
extrapolation. The identifiability is characterized by a novel constraint
describing the linearity assumption of A on Z. Based on this insight, we
propose a method that enforces the linear invariance constraint and can be
combined with any type of autoencoder. We validate our theoretical findings
through synthetic experiments and show that our approach succeeds in predicting
the effects of unseen interventions
- …